Loss Functions, Axioms, and Peer Review

It is common to see a handful of reviewers reject highly novel paper, because they view, say, extensive experiments as far more important than novelty, whereas the community whole would have embraced paper. More generally, disparate mapping criteria scores final recommendations by different major source inconsistency in peer review. In this paper we present framework inspired empirical risk minimization (ERM) for learning community's aggregate mapping. The key challenge that arises specification loss function ERM. We consider class L(p,q) functions, which matrix-extension standard Lp losses on vectors; here choice amounts choosing hyperparameters p and q. To deal with absence ground truth our problem, instead draw computational social identify desirable values Specifically, characterize p=q=1 only these satisfies three natural axiomatic properties. Finally, implement apply approach reviews from IJCAI 2017.